Virtual Worlds and Augmented Reality in Cultural Heritage Applications

Mixed Realities (Milgram & Kishino 1994) and their concept of cyber-real space interplay invoke such interactive digital narratives that promote new patterns of understanding. However, the "narrative” part, which refers to a set of events happening during a certain period of time and providing aesthetic, dramaturgical and emotional elements, objects and attitudes (Nandi & Marichal 2000, Tamura et al 2001) is still an early topic of research. Mixing such aesthetic ambiences with virtual character augmentations (Cavazza et al 2003) and adding dramatic tension has developed very recently these narrative patterns into an exciting new edutainment medium (Lindt 2003). Since recently, AR Systems had various difficulties to manage such a time-travel in a fully interactive manner, due to hardware & software complexities in AR ‘Enabling Technologies’ (Azuma et al 2001). Generally the setup of such systems was only operational in specific places (indoors-outdoors) or with specific objects which were used for training purposes rendering them not easily applicable in different sites. Furthermore, almost none of these systems feature full real-time virtual human simulation. With our approach, based on an efficient real-time tracking system, which require only a small pre-recorded sequence as a database, we can setup the AR experience with animated virtual humans anywhere, quickly. With the interplay of a modern real-time framework for integrated interactive virtual character simulation, we can enhance the experience with full virtual character simulations. Even if the environmental conditions are drastically altered, thus causing problems for the real-time camera tracker, we can re-train the camera tracker to allow it to continue its operation. The proposed set of algorithms and methodologies aim to extend the “AR Enabling Technologies” in order to further support real-time, mobile, dramaturgical and behavioured Mixed Reality simulations, as opposed to static annotations or rigid geometrical objects. Fig. 1 depicts fully simulated virtual humans (skin, clothes, face, body) augmenting a cultural heritage site.